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Researchers at NIET and elsewhere have studied the effectiveness of TAP: The System for Teacher and 
Student Advancement (TAP) in raising student achievement, improving the quality of instruction, and 
increasing the ability of high-need schools to recruit, retain and support effective teachers. This document 
describes some of the most recent results that have emerged from the research on the TAP System to 
date. Data collection and analysis efforts are ongoing, and the findings described here will be updated 
periodically as information becomes available. 
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NIET Mission 


Recognizing that an effective teacher is the most important school-based factor impacting student 
achievement (Darling-Hammond, 2006; Goldhaber, 2002; Hanushek, 2013; Headden, 2014; Rivkin, 
Hanushek, & Kain, 2005; Simon & Johnson, 2013; TNTP, 2012), NIET is committed to ensuring a highly 
skilled, strongly motivated, and competitively compensated teacher for every classroom in America. NIET 
supports states, districts and schools in recruiting, developing, supporting, and retaining high-quality 
human capital in order to raise achievement levels for all students (National Institute for Excellence in 
Teaching, 2015). NIET provides both on-site and online support across multiple aspects of educator 
effectiveness through educator evaluation, professional development and teacher leadership. This 
support is delivered both on-site and online through the comprehensive TAP™: The System for Teacher 
and Student Advancement (TAP) and customizable Educator Effectiveness Best Practices. As of the 2016- 
17 school year, NIET initiatives are impacting over 200,000 educators and more than 2.5 million students. 


TAP: The System for Teacher and Student Advancement Description 

Introduced in 1999, “the TAP System has grown significantly as a comprehensive educator evaluation and 
support model for increasing educator effectiveness” (National Institute for Excellence in Teaching, 2015, 
p. 4). The TAP System creates multiple career paths for teachers, provides ongoing applied professional 
development using a rigorous rubric of evaluation, and provides performance-based compensation to 
teachers and administrators. Each of these core elements is discussed below. For more information, visit 
www.niet.org. 


e Multiple career paths. In TAP schools, skilled teachers have the opportunity to serve as master and 
mentor teachers, receiving additional compensation for providing high levels of support to career 
teachers and increasing instructional effectiveness across the faculty. Master and mentor teachers 
form a leadership team, along with administrators, to deliver school-based professional support and 
conduct evaluations with a high level of expertise. 


e Ongoing applied professional growth. Led by master and mentor teachers, TAP teachers participate 
in weekly meetings where they examine student data, engage in collaborative planning, and learn 
instructional strategies that have been field-tested in their own schools. Teachers benefit from a 
national TAP database of instructional strategies and their colleagues' experiences. Professional 
development continues in the classroom as master teachers model lessons, observe classroom 
instruction, and support teachers’ pedagogical improvement. 


e =Instructionally focused accountability. TAP teachers are observed in classroom instruction several 
times a year by multiple trained observers, including principals and master and mentor teachers, using 
rubrics for several dimensions of instructional effectiveness. Evaluators are trained and certified, and 
leadership teams monitor the reliability and consistency of evaluations in their schools. These 
classroom evaluations are complemented by analyzing student achievement growth, rounding out a 
multi-measure system of teacher evaluation. Evaluation results are used as formative feedback in 
one-on-one mentoring sessions, and guide planning for individualized professional development. 


e Performance-based compensation. TAP teachers have the opportunity to earn annual bonuses based 
on their observed skills, knowledge, and responsibilities, their students’ average achievement growth, 
and school-wide achievement growth. Master and mentor teachers receive additional compensation 
based on their added roles and responsibilities, and principals can earn additional compensation 
based on school-wide achievement growth and other measures of effectiveness. 
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Educator Effectiveness Best Practices Center Description 

The NIET Educator Effectiveness Best Practices Center provides innovative services, support, and solutions 
to schools, districts, and states to improve educator effectiveness. Based on more than a decade of 
experience in schools across the country, NIET works with its partners to redesign educator evaluation, 
deliver effective professional development, implement performance-based compensation systems, and 
train teacher leaders in schools. The BPC offers a network of expert trainers and access to a range of 
innovative Web-based resources and tools. For more information, visit https://nietbestpractices.org. 


NIET Initiative Comparisons and Results 


Since 1999, the TAP System of comprehensive educator effectiveness has operated across multiple states 
in hundreds of schools. TAP has grown steadily in the number of schools participating, the majority of 
which are high-needs schools (over 90% of participating schools). In addition to schools and districts 
implementing the full TAP System, a number of states, districts, and schools are using TAP System core 
elements — in particular, the online portal which houses hundreds of hours of effective teaching across 
subjects and grade levels, the TAP teacher observational rubric, and the certification and recertification 
process for educator evaluators. 


TAP System Evaluation Methods Compared to Traditional Evaluation Methods 

To improve the quality of classroom instruction, the quality of each teacher’s instruction must be 
assessed. Traditional school systems have not been successful at measuring and assessing classroom 
instruction. The New Teacher Project (TNTP) published a revealing report in 2009 showing that schools 
fail to evaluate their teachers in any meaningful way (Weisberg, Sexton, Mulhern, & Keeling, 2009). As 
TNTP reported, most teachers were rated at the very highest levels (replicated as Figure 1 below), despite 
the fact that most schools were not performing at these highest levels on achievement indicators. 


A 2014 report from the National Council on Teacher Quality (NCTQ, 2014) indicated that while 
improvements in educator evaluation have occurred since the 2009 report, still many of the same 
problems persist. Given that differences in teacher effectiveness represent the single most important 
school-related factor affecting student learning, accurately measuring differences in teacher performance 
is critical to the improvement of teaching and learning. 


70% 63% 
60% 
50% 


Percent of 40% 
Teachers 39% 


20% 
10% 3% 5% 
0% =a . | . 


Lowest Ratings Middle Ratings Highest Ratings 


Teacher Evaluation Rating 
Figure 1. Traditional Teacher Evaluation Scores.* 


1 Teacher evaluation in five urban school districts, based on data taken from 
http://widgeteffect.org/downloads/TheWidgetEffect.pdf. Scores on a 3-point and 4-point scale have been 
interpolated to a 5-point scale using a cumulative probability density function based on reported data. 
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In contrast to traditional evaluation methods noted previously in Figure 1, the TAP System has developed 
a comprehensive approach to teacher evaluation and incentives that depends on multiple measures of 
both teaching practice and teaching outcomes. This system provides differentiated feedback for teacher 
improvement, in contrast to the often found inflated ratings found in evaluation systems (see Figure 2). 
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Note: Teacher Skills, Knowledge, and Responsibilities distribution of TAP evaluations using 1-5 scale in half-point 
increments. Figure based on 9,347 teachers and approximately 37,388 observations, 2015-16. 


Figure 2. Observational Ratings of Teachers in TAP Schools Using TAP Teaching Standards Rubric. 


The above ratings are based only on the classroom evaluation component of the TAP System, before 
considering student learning growth measures. Teachers are observed several times a year by multiple 
trained and certified raters who consider 19 areas of effective instructional practice. These observers use 
a multi-dimensional, research-based set of standards and rubrics that are fair, transparent, and 
curriculum-independent. Results are provided immediately as feedback to the teacher in post- 
observation mentoring sessions. The scores from all observations of these 19 classroom indicators are 
combined with seven responsibility indicators at the end of the school year to create an overall Skills, 
Knowledge, and Responsibilities (SKR) score for each teacher. On a scale of 1.0 to 5.0, 1 represents 
unsatisfactory performance on a certain standard, 3 represents proficiency on that standard, and 5 
represents exemplary performance on that standard. Teachers earn scores in increments of 0.5. 


TAP Teachers Demonstrate Consistent Improvement across Time 

TAP results show a steady improvement in observed skills during the course of the school year. Figure 3 
shows recent improvement for teachers in TAP nationally. This shows the improvement in instructional 
quality scores over a two-year period. In the data shown, despite a slight dip over the summer, teachers 
demonstrated an overall path of improvement that continued over both years. This graph is based ona 
sample including all TAP schools during the years 2014-2016. 


We tracked a cohort of 2,368 career teachers through observations grouped into six periods in fall, winter 
and spring of the 2014-2015 and 2015-2016 school years. The cohort was composed of teachers working 
in TAP schools both years, with observations in each of the observation periods. Teachers present in only 
one school year or who lacked an observation in one of the quarters were excluded from the sample. 
Including teachers present in only some of the periods would have complicated efforts to compare the 
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level of instructional quality at different time points, since each average could reflect substantially 
different groups of teachers. 
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Figure 3. Teacher Instructional Improvement across Time. 


Within the TAP System, improvements to teacher instructional practices translate into gains in student 
achievement. More importantly, these improvements increase over time. 


90% 
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Schools Achieving 30% 79% 
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88% 
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Note: Figure created with raw school-level, composite math and reading score data provided by SAS® for control 
(n=3,870) and TAP schools (n=353). 


Figure 4. Student Achievement Growth by TAP System Status. 


TAP Impact in Schools and Districts 

A notable success of the TAP System is the expanding preponderance of performance data that comes 
from examining the impact of the system across multiple locations and conducted by different researchers 
using varied methodological frameworks (Algiers Charter School Association, 2011; Barnett, Hudgens, 
Logis, & Alexander, 2016; Barnett, Hudgens, & Alexander, 2016; Barnett, Rinthapol, & Alexander, 2015; 
Barnett, Rinthapol, & Hudgens, 2014; Barnett, Wills, Hudgens, & Alexander, 2015; Buck & Coffelt, 2013; 
[eee fl 
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Daley & Kim, 2010; Hudson, 2010; Schacter & Thum, 2005; Schacter et al., 2002; Schacter, Thum, 
Reifsneider, & Schiff, 2004; Solmon, White, Cohen, & Woo, 2007). The next section discusses several of 
these studies demonstrating the impact of the TAP System. Following this general review, two new 
studies, a project spotlight, and results from the 2016 TAP Attitude Survey are presented. 


Results from Prior Selected Studies of TAP’s Impact 


Mann, Leutscher, and Reardon (2013) examined the impact of the TAP System across fifteen schools 
in Louisiana. In order to determine impact, a one-to-one nearest-neighbor matching algorithm with 
replacements was created to find a comparison school for each TAP school. Based on the propensity 
scores computed using the selection model, the algorithm chooses the non-TAP school with the 
propensity score closest to the propensity score of the TAP school. There was no significant difference 
between the TAP schools and their matched comparison schools in the pretreatment year, t(26) = 
0.080, p > 0.05. However, results of the study indicate that in the four primary subjects assessed, there 
was a significant effect in favor of the TAP schools for ELA: F(1, 6421) = 6.334, p = 0.012; Mathematics: 
F(1, 6421) = 86.386, p < 0.001; Science: F(1, 7084) = 31.792, p < 0.001; and Social Studies: F(1, 7085) 
= 87.411, p < 0.001]. Further, the study examined the impact of the TAP System across time to find 
that the TAP schools significantly outperform comparison schools, F(1, 24) = 5.30, p < 0.05. The study 
also found that 92% of teachers reported that TAP made a positive difference on student achievement 
in their school and 91% reported that the AYP status was improved as a result of the TAP System. 


In 2010, Hudson examined the effect of the TAP System on student achievement across 151 schools 
in 10 states. Hudson used a statistical control matching method to ensure that the TAP schools and 
the comparison schools were equivalent prior to the intervention being implemented. Hudson also 
used a differences-in-differences approach to further account for any differences between the groups 
and to ensure that the evaluation was able to isolate the impact of the program. Results of the study 
indicate that students in TAP schools outperformed students in comparison schools by approximately 
0.15 standard deviations in mathematics, and smaller effects but in favor of the TAP schools in 
reading. Hudson explains these findings in context to other education interventions by noting that 
“the estimated effect of TAP on mathematics achievement is more than twice as large [as class size 
reduction effects]” (p. 28). 


In 2007, Solmon et al. analyzed the impacts of the TAP System in terms of value-added gain scores 
across 650 classrooms in six states, including 61 TAP schools and 285 control schools. Researchers 
analyzed the student achievement gains at two levels of comparison—teacher-to- teacher and school- 
to-school. To evaluate TAP teachers (and similarly in evaluating TAP schools), researchers calculated 
the effect of each teacher on student progress as assessed by the difference between the actual 
average scores of the teacher’s students and the expected average scores of those students (as 
derived from previous scores). Through this process, researchers created a statistical control group 
for the TAP teachers based on performance. Results of the study indicate that in every state more TAP 
teachers demonstrated statistically significant at or above average amount of student achievement 
growth than control group teachers. Further, TAP schools outperformed their controls in 57% of the 
categories in math and in 67% of the categories in reading. 


In 2004, Schacter et al. examined the impact of the TAP System across 11 schools. This was a follow- 
up study of their 2002 study in which the authors conducted a statewide cluster analysis to analyze 
the growth in achievement of students and found that TAP schools made significantly higher 
improvements in student achievement gains. In the follow-up study, the same cluster level analysis 
with multi-level multivariate analyses were employed using all available covariates to compare growth 
between the TAP and control schools. Results from the study indicate that 65% of the TAP schools 


NATIONAL INSTITUTE FOR EXCELLENCE IN TEACHING 7 


outperformed their matched controls in reading, language, and mathematics achievement, with the 
magnitude of change ranging from 6% to 46%. The teacher satisfaction component of this study 
indicated strong support for the four core principles of the TAP System. 


e In 2014, Barnett, Wills, and Kirby evaluated the impact of the TAP System across 66 schools in 
Louisiana using two rigorous analytic strategies. First, the authors employed a linear regression to 
compare 2012-13 Assessment Index (Al) performance of the 66 TAP schools and non-TAP schools 
statewide, controlling for prior (2010-11) achievement, percentage of students receiving free or 
reduced-price lunch, school configuration, school size (number of students), and percentage of English 
language learners. Controlling for the covariates, implementation of the TAP System showed a 
significant positive effect on 2012-13 achievement: the 66 TAP schools scored 3.7 points higher on 
average than non-TAP schools (p < .01). Second, the authors compared the 66 TAP schools with a 
propensity score matched group of non-TAP schools. The average 2012-13 Al for TAP schools (64.45) 
was 5.47 points greater than the average for the matched controls (58.98). 


e In their 2014 study, Barnett and Hudgens drew upon a sample of 12,095 teacher-level records, 
representing 413 schools in 10 states, to examine teacher retention rates in schools that implemented 
the TAP System during the 2010-11, 2011-12, and 2012-13 school years. Counter to national trends, 
the authors found that TAP System schools retain 14% more teachers than similar high-needs 
comparison schools and 10% more than the national average. Furthermore, the results indicate 
teachers retained in TAP schools become more effective over time as measured by observational and 
value-added scores. 


e =Barnett and Wills (2016a) examined the impact of the TAP System on Black-White and Hispanic-White 
achievement gaps in Indiana schools. First, the authors compared achievement gaps in 32 TAP schools 
that began implementing TAP in school year 2010-11 with other Indiana schools. Study results indicate 
TAP schools have narrowed the Black-White gap in ELA and Hispanic-White gap in ELA and math more 
than other Indiana schools have. Second, the authors selected propensity score matched control 
schools separately for 1) the 20 TAP schools reporting data for Black and White students in 2009-10 
and 2013-14, 2) the 24 TAP schools reporting data for Hispanic and White students in 2009-10 and 
2013-14. Study results indicate TAP schools outperformed their matched schools; while achievement 
gaps at the TAP schools narrowed, most gaps at the matched schools widened. 


e Barnett and Wills (2016b) investigated the stability over time of classroom observation scores from 
the TAP System rubric, as well as the relationships between TAP observational scores and value-added 
measures. The correlations between average SKR scores in the 2010-11, 2011-12, and 2012-13 school 
years were highly significant and large, particularly for consecutive school years. Further, the 
examination of correlations between average SKR scores and classroom value-added scores in the 
2010-11, 2011-12, and 2012-13 school years revealed highly significant correlations. To put these 
findings into context, the authors examined the reported correlations across other observational 
measures used in large-scale research studies and discovered these correlations exceed or match the 
range reported in recent literature — evidence that the TAP observation rubric measures aspects of 
teacher practice that contribute to improved student test performance. 


Results from New Selected Studies of TAP’s Impact 


Impact of TAP on Educator Effectiveness in TIF-4 Sites 

With support from three federal Teacher Incentive Fund (TIF-4) grants, NIET partnered with five charter 
schools in Minnesota, six schools across two districts in lowa, and 11 schools across five districts in 
Tennessee. The first year (2012-13) was a planning year; implementation of the TAP System began during 
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the 2013-14 school year. In what follows, we provide results from evaluating the impact of TAP 
implementation on teacher and administrator effectiveness in these sites. 


To measure teacher effectiveness, the authors drew from Skills, Knowledge, and Responsibility (SKR) 
scores and classroom value-added (CVA) scores. To be considered effective, teachers must have received 
an SKR score of at least 2.5. For classroom value-added, teachers must have obtained a score of 3.0 or 
higher, a value that translates into at least one year’s expected growth of students. To measure 
administrator effectiveness, the authors drew from TAP Leadership Team (TLT) scores, Vanderbilt 
Assessment of Leadership in Education (VAL-ED; for schools in Minnesota and lowa) and Tennessee 
Instructional Leadership Standards (TILS; for schools in Tennessee). To be considered effective, principals 
must have received a TLT, VAL-ED, or TILS score of at least 3.0. 


Minnesota 

In Minnesota, examination of the overall SKR mean indicates improvement in pedagogical practices over 
time; the average SKR score increased from 2013-14 (M = 3.18, SD = 0.45) to 2015-16 (M = 3.22, SD = 
0.48). As shown in Figure 5, the percentage of teachers meeting the CVA effectiveness criteria also 
substantially increased from 2013-14 to 2015-16. 


100% 95% 

80% Tes 

‘0 
68% 
Percent Meeting 60% 
Effectiveness criteria 
(CVA 3.0 or higher) 40% 
20% 
0% 
2013-14 2014-15 2015-16 


Figure 5. Minnesota TIF-4 Percent of Teachers Meeting CVA Effectiveness Criteria of 3.0 or Higher, by Year. 


As teacher effectiveness improved, so did principal effectiveness; the average TLT score increased from 
3.49 (SD = 0.35) in 2013-14 to 3.56 (SD = 0.63) in 2015-16. 


lowa 

Teachers and administrators in lowa also showed increases in effectiveness over time. As shown in Figure 
6, the average SKR score for all teachers improved from 2013-14 to 2015-16. Because the lowa TIF-4 grant 
has a STEM component, the authors examined the effectiveness of STEM teachers separately. The average 
SKR score among STEM teachers also consistently improved over time. 
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Figure 6. lowa TIF-4 Mean SKR Scores for All Teachers and STEM Teachers, by Year. 


Regarding classroom value-added scores, from 2013-14 to 2015-16, there was a 22% increase in the 
percent of teachers achieving at least one year’s expected growth of students in their classrooms. Among 
STEM teachers, there was a 39% increase in the percentage of effective teachers from 2013-14 to 2015- 
16. 


The average TLT score of principals in lowa also improved, from 2.64 (SD = 0.38) in 2013-14 to 3.29 (SD = 
0.76) in 2015-16. As shown in Figure 7, the most profound finding was the percent of principals meeting 
the VAL-ED effectiveness criteria, which increased from 0% in 2013-14 to 100% in 2015-16. 
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Figure 7. lowa TIF-4 Percent of Principals Meeting VAL-ED Effectiveness Criteria of 3.0 or Higher, by Year. 


Tennessee 

Building on the 2016 study conducted by Barnett and Hudgens (Barnett & Hudgens, 2016), the current 
study examined the impact of TAP implementation on teacher and administrator effectiveness in 
Tennessee. All Tennessee TIF-4 teachers continued to meet the SKR criteria of 2.5 or higher in the 2015- 
16 school year. Examination of the overall SKR mean indicates improvement in pedagogical practices over 
time; the overall SKR mean increased from 2013-14 (M = 3.56, SD = 0.40) to 2015-16 (M = 3.91, SD = 0.43). 
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The mean SKR score for each teacher role increased over time. Further, at the three examined time points, 
master teachers scored higher on average than mentor teachers, who, in turn, scored higher on average 
than career teachers (see Figure 8). 
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Figure 8. Tennessee TIF-4 Mean SKR, by Year and by Career Path. 


Examination of TLT scores suggests improvement in principal effectiveness over time; from 2013-14 to 
2015-16, the average TLT score increased from 2.91 (SD = 0.54) to 3.82 (SD = 0.50). Additionally, the 
percentage of administrators meeting the TILS criteria increased over time, such that by 2015-16 100% of 
principals obtained a score of 3 or higher. 


The TAP System in Texas 

The state of Texas began implementing the TAP System in 2005 with a pilot of three schools, and since 
that time, the number of TAP schools has expanded. As the year 2010 marked the biggest infusion of TAP 
into Texas with the support from a federal Teacher Incentive Fund (TIF-3) grant, we examined the impact 
of TAP in Texas schools that 1) implemented the system between 2010-11 and 2014-15 for at least two 
years in a row and 2) had school performance data available for one year prior to TAP implementation 
(“pretreatment year”). Based on this criteria, the final sample included 46 schools serving K-8 grade 
students. Using this sample, we tested whether TAP schools outperformed similar non-TAP schools in 
math and reading state assessments. 


We drew school performance data from the Texas Education Agency (TEA) website. School academic 
performance was measured using school-level average scale scores in math and reading on the state 
assessments. To assess performance we computed the average of scale scores across grades for each 
school, weighted by the number of students in each grade taking the test. As TEA replaced its system of 
testing and evaluating students from Texas Assessment of Knowledge and Skills (TAKS) to the State of 
Texas Assessment of Academic Readiness (STAAR) beginning in the 2011-12 academic year, the resulting 
weighted scale scores were then transformed into standardized scores for each school to enable 
comparison across different tests and academic years. 


In order to determine impact, we performed a one-to-one propensity score analysis (Rosenbaum & Rubin, 
1983; 1985) to find a comparison school for each TAP school that had similar school performance in the 
pretreatment year and the same school configuration. Using the sample of 46 TAP schools and 46 matched 
schools in Texas, Figure 9 shows that over the duration of TAP implementation, TAP schools outperformed 
matched schools by an average of 0.12 standard deviation in math and reading assessments combined. 
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Figure 9. School Performance Gain in Standard Deviation Units in 46 TAP Schools versus 46 Matched Schools. 


In a second analysis, we refined the sample by selecting a subset of TAP schools that met the following 
criteria: 1) had teachers who reported strong levels of support on key elements of TAP and 2) did not 
experience any dramatic changes in school demographics. These criteria yielded 29 TAP schools. Figure 
10 shows that throughout the implementation of the TAP System, these 29 TAP schools outperformed 
matched schools by an average of 0.38 standard deviation (SD = 1.20, t(28) = 1.69, p = 0.10) in math and 
reading assessments combined. 


0.40 
0.35 
0.30 
0.25 
School Performance 9.20 


Gain in Standard 0.15 
Deviation Unit 0.10 


TAP Schools 


m Matched Schools 


0.05 
0.00 
-0.05 
-0.10 


-0.04 


Figure 10. School Performance Gain in Standard Deviation Units in 29 TAP Schools versus 29 Matched Schools. 


Project Spotlight: Impact of NIET and Institutions of Higher Education 


Partnerships 


Institutions of higher education (IHE) and colleges of education in particular continue to face myriad 
challenges regarding their connection to practicing teachers (Greenberg, Walsh, and McKee, 2015). Two 
innovative collaborations between non-profit, university, and district partners are working to address this 
challenge by connecting the principles of a proven district-based educator effectiveness model with initial 
teacher preparation. NIET is partnering with two influential university teacher preparation programs — the 
Mary Lou Fulton Teachers College at Arizona State University (ASU), with 21 Arizona partner school 
districts (65 schools), and the College of Education at Texas Tech University (TTU), with five Texas partner 
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districts (18 schools). Supported by a U.S. Department of Education Supporting Effective Educator 
Development (SEED) grant, these partnerships are connecting the pipeline of teacher preparation to 
NIET’s educator effectiveness model TAP™: The System for Teacher and Student Advancement. The 
revised pipeline begins with a rigorous recruitment process, proceeds to reformed curricula and extensive 
clinical preparation, and culminates in a more seamless transition to effective in-service teaching. While 
infusing the TAP Teaching Standards Rubric into the teacher preparation program, these partnerships 
provide teachers and school leaders with advanced training and professional development to enhance 
their effectiveness. Evaluation of the impact of this three-way partnership has revealed successes in the 
following five areas: 


1. High graduation rate. 

As shown in Figure 11, the graduation rate of teacher candidates in NIET-IHE partnership ranged from 86% 
to 100%, whereas, on average, only 38% to 40% of students who enrolled in four-year postsecondary 
institutions nationwide graduated within four years. 
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Note: Nationwide graduation rates were taken from National Center for Education Statistics data for 2005 through 
2009 starting cohorts https://nces.ed.gov/programs/digest/d16/tables/dt16_ 326.10.as 


Figure 11. Graduation Rate Comparison between NIET-IHE Partnership and Nationwide, By Year. 


Further, 155 of 164 (94%) total teacher candidates who have graduated from NIET-ASU and NIET-TTU 
partnerships, 90% obtained employment upon graduation, with 85% of them employed in Title 1 
schools. 


2. Recruitment of high quality teacher candidates. 

To evaluate the impact of recruitment, we compared admitted and non-admitted candidates in the NIET- 
ASU partnership on their incoming GPA and the National Evaluation Series (NES) exam scores. Figure 12 
shows that admitted candidates significantly outperformed non-admitted candidates in cumulative GPA 
(t(408) = 3.65, p< .001) and the NES exam (t(47.39)? = 5.34, p< .001). Further, compared to other teacher 
candidates within similar programs at ASU, the percentage of teacher candidates in this partnership who 
completed the program requirement and passed professional knowledge and subject knowledge 
certification exams were 14% to 32% higher. 


? Based on Satterthwaite degrees of freedom 


NATIONAL INSTITUTE FOR EXCELLENCE IN TEACHING 13 


Non- 


3.35 3.40 3.45 3.50 3.55 3.60 200 210 220 230 240 250 


Cumulative GPA National Evaluation Series (NES) Score 


Figure 12. GPA and Certification Exam Scores for Candidates in NIET-ASU Partnership, By Admission Status. 


3. Improving teacher candidates’ effectiveness. 

Throughout their residency period, teacher candidates were evaluated four times (twice per semester) 
through a rigorous performance assessment process using the TAP Teaching Standards Rubric. The 
evaluation process for teacher candidates mirrors the formal evaluation conducted for in-service TAP 
teachers. To be considered effective, teacher candidates must have received a score of 2.5 or higher on 
the TAP Rubric. Figure 13 shows that on average, teacher candidates exceeded the standard for effective 
instruction by the second observation, and graduated with an average score above 3.0, demonstrating 
strong instructional skills. Analysis using a multilevel longitudinal model further suggests that the 
improvement in teacher candidates’ effectiveness over time was statistically significant (B = 0.45, SE = 
0.04, p < .001 for linear growth and B = -0.03, SE = 0.01, p < .01 for quadratic growth). 
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Note: Figure based on 109 teacher candidates in NIET-IHE partnerships who had a score at each performance 
assessment. 


Figure 13. Average TAP Rubric Scores of Teacher Candidates in NIET-IHE Partnerships during Clinical Training. 


4. Promoting school academic growth. 

NIET’s partnership with Texas Tech University includes providing teachers and school leaders with content 
knowledge expertise in the areas of STEM, Literacy and Leadership. The entire staff at Slaton Junior High 
in Slaton ISD began participating in this advanced training in fall 2015. To determine impact, we drew data 
SSS | 
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from the TEA website and compared the percent of students meeting or exceeding the STAAR Level II 
Satisfactory Standard in Slaton Junior High two years before (2012-13) and one year after (2015-16) 
participating in the advanced training. Figure 14 below suggests that Slaton Junior High is progressing 
towards closing the gap with the state average in all subjects, and especially in math. The school also 
showed substantial improvement from 2012-13 to 2015-16 as reflected in the school accountability 
reports obtained from the TEA website: from being rated as “improvement required” with no distinctions 
in 2012-13 to being rated as “met standard” with four distinctions in 2015-16. 
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Figure 14. Average School Performance in Slaton Junior High and Statewide for All Subjects and Math, By Year. 


5. Increasing school leader effectiveness. 

Through the SEED grant, TAP school leaders in Texas were provided with technology-enabled, ongoing 
competency-based feedback and shaping on TAP school processes. To assess the effectiveness of this 
added support in improving school leader effectiveness, each year we rated principal effectiveness based 
on mid-year formative assessment of the Leadership Team Meeting and end of year School Review 
Principal Leadership Score. As shown in Figure 15, we found a 56% increase in the percentage of principals 
rated as proficient from 2013-14 to 2015-16. 
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Figure 15. Percent of Principals Rated as Proficient, by Year. 
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National TAP Attitude Survey — 2016 


Teacher Results — 2016 TAP Attitude Survey 

Collegiality. In the 2016 TAP national survey of teacher attitudes (n = 6,416 teachers), we find evidence of 
a high degree of collegiality in TAP schools. Specifically, 94% of teachers in TAP schools agree with 
statements reporting a high level of collegiality in their schools, and over 70% report strong agreement. 
As one teacher shared: 


“TAP has encouraged an atmosphere of teamwork and has created an environment where all teachers 
strive for better results. | have never been a part of a work environment that was more driven.” 


This evidence for collegiality has been remarkably high over the last decade, as shown in Figure 16°, which 
indicates that TAP System schools consistently have a collaborative and collegial environment. 


Collegiality 


0% 20% 40% 60% 80% 100% 
m= Moderate Support ™ Strong Support 


Figure 16. Level of Reported Support for Collegiality from Teachers Nationwide. 


Beyond the overall high levels of reported collegiality within TAP schools, the levels of support for the 
specific elements of TAP including multiple career paths, ongoing professional growth, instructionally 
focused accountability and performance-based compensation are also high and growing, as shown in 
Figures 17-20. 


Multiple career paths. A growing number of teachers report that the creation of teacher leader roles in 
their school has a positive impact on student achievement and school goals. Teacher leaders and 
administrators form a leadership team that articulates school goals and supports each teacher in 
developing and achieving their own instructional goals based on their skills and their students’ needs. 
Teacher leadership roles also provide a pathway for teachers to make a greater contribution to the 
instructional excellence of a school without leaving the classroom. As one teacher testified: 


“TAP has allowed us to grow into better educators through the constant feedback that we receive from 
mentor and master teachers.” 


3 The five dimensions represented in this and the next four figures are constructed from multiple teacher survey items 
using factor analysis. Most items in the survey are based on a 5-point Likert scale indicating agreement (1=Not at all and 
5=Very Much). For reporting purposes on the four TAP elements and collegiality, the results are presented as Moderate 
(weighted average of 3 on the items for that factor) and Strong (weighted average of 4 or 5 on the items for that factor). 
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Multiple Career Paths 
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Figure 17. Level of Reported Support for Multiple Career Paths from Teachers Nationwide. 


Ongoing applied professional growth. In TAP schools, master and mentor teachers lead weekly cluster 
group meetings where they examine student data, engage in collaborative planning, and discuss 
instructional strategies that have been field-tested in their own schools. Teachers benefit from access to 
a national TAP database of instructional strategies and their colleagues' experiences. Professional 
development continues in the classroom as master teachers model lessons, observe classroom 
instruction, and support teachers’ pedagogical improvement. As one teacher stated: 


“To me, the best part of TAP is cluster. It gives me an opportunity to work on areas that will help my 
specific students for the year.” 


Figure 18 demonstrates the strong level of support for the professional growth that occurs in TAP schools, 
and further shows the sustained high level of support. 


Ongoing Applied Professional Growth 
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Figure 18. Level of Reported Support for Ongoing Applied Professional Growth from Teachers Nationwide. 


Instructionally focused accountability. TAP teachers are observed in classroom instruction multiple times 
a year by multiple trained observers, including principals and master and mentor teachers, using rubrics 
measuring indicators of instructional effectiveness. Evaluators are trained and certified, and leadership 
teams monitor the reliability and consistency of evaluations in their schools. These classroom evaluations 
are complemented by analyzing student achievement growth, rounding out a multi-measure system of 
teacher evaluation. Evaluation results are used as formative feedback in one-on-one mentoring sessions, 
and guide planning for cluster group meetings. As one teacher wrote: 


“TAP has allowed me to reflect more on my teaching practices and student learning. | feel | have grown 
more this year than in my previous years of teaching. The rubric helps set great expectations for 
teachers, which leads to greater teacher and student success and motivation in the classroom.” 

Figure 19 illustrates the strong level of support reported by TAP teachers across the nation. 
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Instructionally Focused Accountability 
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Figure 19. Level of Reported Support for Instructionally Focused Accountability from Teachers Nationwide. 


Performance-based compensation. TAP teachers have the opportunity to earn annual bonuses based on 
their observed skills, knowledge and responsibilities, their students’ average achievement growth, and 
school-wide achievement growth. Master and mentor teachers receive additional compensation based 
on their added roles and responsibilities, and principals can earn additional compensation based on 
school-wide achievement growth and other measures of effectiveness. As one teacher reported: 


“I love that | can receive an extra financial reward for my hard work as a teacher. The staff look forward 
to the TAP payout checks.” 


Figure 20 demonstrates the level of support for the performance-based compensation system from TAP 
teachers. Levels of support for performance-based compensation in TAP schools has increased 
significantly over time, from 49% in 2005 to 79% in 2016. 


Performance-Based Compensation 
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Figure 20. Level of Reported Support for Performance-Based Compensation from Teachers Nationwide. 


Administrator Results — 2016 TAP Attitude Survey 

The above substantially positive results from the TAP teacher survey are echoed by the 2016 TAP principal 
survey (n = 320 administrators). When asked about the impact TAP has had on their school, one principal 
wrote: 


“Teachers feel more confident in the classroom as a result of the professional development and 
understanding of the rubric. | have asked several teachers to compare their teaching now with how they 
taught prior to TAP. All of the teachers explained how their teaching is much more effective now than 
in previous years. None will go back to teaching the ‘old way.’” 
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As shown in Figure 21, principals have overwhelmingly reported that TAP has a positive effect on 
instructional practices, collegiality, student achievement, and teacher effectiveness. Further, as noted on 
Figure 22, a significant majority of principals (88%) agree that TAP helps retain effective teachers. 
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Figure 21. Administrator Survey Results on TAP Outcomes in their Schools 
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Figure 22. “The implementation of TAP has helped retain effective teachers at my school.” 


NATIONAL INSTITUTE FOR EXCELLENCE IN TEACHING 19 


Upcoming Projects 


5,000 Strong: Teacher’s Responses to a New Educator Effectiveness and Support System 
The annual TAP Teacher and Administrator Attitude Survey has collected responses from approximately 
25,000 teachers over the last decade on teacher attitudes and job satisfaction. The survey contains 
questions that assess the perceptions of the implementation of TAP at the school and teachers’ and 
administrators’ overall job satisfaction. Past summaries have provided a glimpse into those responses 
aggregated across all respondents and comparing early adoption years to more current years. In this study, 
the authors examine survey responses from the 2015-2016 school year, which includes over 6,000 
teachers from 222 schools in 9 states. Specifically, the authors focus on 1) whether teachers are supportive 
of the TAP System’s core elements and whether their opinions changed over time, 2) whether teachers 
are satisfied with the implementation of the TAP System, and 3) teachers’ reported impacts of the TAP 
system. 


Stayers, Movers, and Leavers: Impact of an Educator Effectiveness System on Retaining 


Administrators Nationwide 

In 2014, Barnett and Hudgens released a report examining teacher retention rates in schools that 
implemented the TAP System. Recognizing the role administrators play in the successful implementation 
of any reform effort, in a follow-up piece, the authors use school rosters, administrator effectiveness data, 
and school data to investigate the retention and impact of administrators who work in TAP System 
schools. Specifically, the authors examine retention rates for multiple school years across three types of 
administrator groups, those who: 1) worked continuously at the same TAP school (“TAP school stayers”); 
2) transferred from one TAP school to another TAP school (“TAP school movers”); and 3) left TAP schools 
altogether (“TAP school leavers”). Additionally, the authors analyze characteristics of these three 
categories of administrators. 


Conclusion 


The TAP System stands out because of its more than a decade-long track record of growth and success in 
raising student achievement in high-need schools. The research evidence also reveals several key reasons 
for TAP’s positive impact: an evaluation system capable of differentiating teacher performance levels and 
providing detailed feedback for improvement, ongoing professional growth in classroom practice using 
student and teacher data to guide improvement, recruitment and retention of effective teachers, and the 
creation of a challenging, rewarding, and collegial environment focused on high-quality instruction and 
student learning. 
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